The capacity of time-delay recurrent neural network for storing spatio-temporal sequences

We investigate the capacity of a type of discrete-time recurrent neural network, called timedelay recurrent neural network, for storing spatio-temporal sequences. By introducing the order of a spatio-temporal sequence, the match law between a time-delay recurrent neural network and a spatio-temporal sequence has been established. It has been proved that the full order time-delay recurrent neural network of l-step feedback is able to learn and memorize (or store) any bipolar (or binary) spatio-temporal sequence of the order k if k6 l, and that the asymptotic memory capacity of the 1rst-order time-delay recurrent neural network of one-step feedback is not less than C1(n) = n+1, where n is the number of processing neurons in the network. Moreover, we substantiate the theoretical results by simulation experiments. c © 2003 Elsevier B.V. All rights reserved.